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Description 

RELD OF THE INVENTION 

5 [0001 ] This invention relates to an automatic tone adjustment method for computer processing of digital images which 
have a dynamic range larger tiian the intended output display media, such as photographic paper, monitor display, and 
transparent film. 

BACKGROUND OF THE INVENTION 

10 

[0002] Many image capture devices can record a signal dynamic range that is wider than a typical display de^e can 
render effectively. For example, consumer color negative films can record scene luminance range of 1000:1 , but photo- 
graphic color papers can only render a luminance range from 50:1 to 100:1 . Another example is the storage phosphor 
used in digital radiography, which can capture an x-ray exposure dynamic range of 10.000:1 , but a radiographic film as 
15 an output medium can render only a email fraction of it (about 20:1) in good contrast In both examples, wide dynamic 
range images will suffer very significant loss of details in the highlight and the shadow, when they are displayed on their 
intended output madia. Several inventions have been disclosed for solving such problems. 

[0003] The best known method is the unsharp masking and its variations. U.S. Patent No. 5.012,333 and U.S. Patent 
No. 4.812,903 teach methods of decomposing an input image into a low-frequency component and a high-frequency 

20 component. By modifying the low-frequency component with a nonlinear tone scale curve and amplifying the high-fre- 
quency component (unsharp masMng), one can combine the two modified conponenls back to form a new image 
which has a smaller dynamic range and yet retain most image details. U.S. Patent No. 5,608.813 and U.S. Patent No. 
5.454,044 teach a similar method, but formulates it in spatial ctomain. The low-frequency (unsharp) component is 
obtained by averaging image values over a local window. The low-frequency component is then used in a monotonically 

25 decreasing function to determine how much gray value is Id be added or substracled fnom the original image. In 
essence, the method used the unsharp version of the original image to mask off the dynamic range in the input signal. 
These tinree methods are mathematically similar to the well known unsharp masking technique in photography, and ail 
of tiiem suffer from the same drawback of creating dark and bright banding around high-contrast edges as the unsharp 
masking techniqure does. The cause of this artifact Is Illustrated in Rgure 1. The unsharp (bw-frequency) component 

30 rounds off a sharp edge signal to produce a smoothed version. The difference signal (the high-frequency component) 
between the original image and the unsharp version thus contains an overshoot and an undersfioot around a sharp 
edge. When the difference signal (the high-frequency) is added back to the dynamic-range compressed low-frequency 
component, the uncompressed or amplified overshoot and undershoot signals around a sharp edge will create a bright 
band and a dark band near the edge. This artifact is more visible near high contrast edges. TTiis artifact is called the 

35. edge banding artifact. The edge banding artifact is a very objectionable artitect in consumer images and a potentially 
dangerous artifact in digital radiography. In order to deal with this banding problem. U.a Patent 5,467.404 teaches a 
method of deconnposing an image into multiple resolutions and using a predetermined nonlinear amplitude compres- 
sion function for the high frequency component in each resolution. If ttia high frequency amplitude Is small, ft is passed 
unchanged or even amplified. If the high frequency amplitude is large, then it is attenuated. The assumption is that 

40 whenever the signal amplitude gets large, regardless of its cause, one should reduce its amplitude so that the overshoot 
and tine undershoot near a sharp edge can be properly suppressed. A major deficiency of the method Is that the ampli- 
tude in the each resolution alone can not be used to adequately klentily whether the large amplitude is caused by a high 
contrast edge or an image texture. Wrthout making the distinction, fine details and textures of high amplitudes will also 
be suppressed, resulting loss in details. A second msgor deficiency is that a predetermined nonlinear amplitude com- 

4S pression can not adaptively adjust the right amount of compression needed for edges In each resolution of Image 
details at different spatial locations. Since every input image has a different dynamic range and thus requires varying 
amount of dynamic range compression. The amount of compression has to be coupled wittn how the algorithm adjusts 
the amplitude compression for each high frequency component This need was mentioned in U.S. Patent No. 
5.493,622, but it did not teach how to compute tiie necessary tone compression curve, nor did it teach how the tone 

so compression curve shouki t>e used to deal with the edge barxiing artifacts. 

[0004] A third deficiency is that tiie frequency deconnposition as described in U.S. Patent 5,467,404 has to sum up 
pixelwise (after interpolation to the required number of pixels) to the original image when no modification is pertormed 
on any of the signal amplitudes in the decomposition. This constraint limits the types of detailed signals that can be 
extracted from the decomposition. 

ss [0005] Anotiier related approach to the frequency decomposition and modification method is the wavelet transform 
and inverse transform. f=or example. J. l-u. D.M. Healy, Jr., and J.B. Weaver disclosed such a mettiod in the paper "Con- 
trast enhancement of medical Images using multiscale edge representation," Opticai Engineenng, 33, 7, 2151-2161 , 
1994. 
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[0006] A Bimilar invention was also disclosed in U.S. Patent 5.71 7,791 . In these two prior arts, the goal was to adjust 
local image co ntrast through a multiresoluition analysis. Although the multiscale edge structure used by both prior arts 
Is Indeed flexible and its proper use allafws one to alleviate the third deficienc/ mentioned above, both prior arts still suf- 
fer the first two major deficiencies poirrted out earlier: there was no explicit interaction between edges from different 

5 spatial scales and no explicit method to compute how much contrast should be adjusted. Bolti problems are solved in 
this new invention. A contrast gain -control (CGC) module in this new invention derives gain factors from edges of all 
scales and apply them to control how much edge Information to add backto the image information at each scale. A tone 
scale Gun^e (TSC) nnodule in this new Inventbn explidtiy maps the Input dynamic range to the dynamic range of the out- 
put medium. The details of these will be explained shortiy in the following sections. 

10 [0007] A different idea to deal with the edge banding artifact Is disclosed in U.S. Patent No. 5,471 ,967 and U. S. Patent 
No. 5J66,870, where the unsharp (low frequency] component is calculated from a weighted average of a predeter- 
mined mask. The weighting is chosen so tiiat less weight is applied to a given peripheral pixel in the mask if its absolute 
difference from the center pixel is large. The weighting is also chosen so that less weight is applied to a peripheral pixel 
If it is located farther away from the center pixel. By such a differential weighting scheme, pixels across a high contrast 

75 edge are not averaged togettier and therelbre, tiie unsharp component has relatively sharp high contrast edges. If the 
image detail (high frequency) component is calculated by subtracting the unsharp component from the original image 
signal, then the image detail (high frequency) component will not have the overshoots or undershoots near a high con- 
trast edge. Therefore, there will be no edge banding artifacts when the unsharp component and the image detail com- 
ponent are combined. This method is fairly effectiva However, there are two major drawbacks in such a method. The 

20 first drawback is that it is very slow, because the weighted averaging operation is not a regular convolution operation. 
The seoond drawtsack is that the method can only deal with high contrast edges that have a short transition distance. If 
a luminance transition occurs oontinously over some longer distance, the absolute difference between the center pixel 
and its neighboring pixels will not be large, and therefore the weighting win be relatively unHbrm over the transition. The 
unsharp component will then be a blurred version of the original Image and the image detail component will contavi 

25 overshoots and undershoots which, when added back or enhanced, will be visible in the output image. 

SUMMARY OF THE INVENTION 

[0006] The present invention is directed to overcoming one or more of the problems set forth above. Briefly summa- 
30 rized, according to one aspect of the present invention, the invention resides in a method of reducing the dynamic range 
of an input image for effectively rendering the input image on an output display medium, the method comprising the 
steps of (a) extracting detail signals and coarse signals from the input image; (b) generating contrast gain-control sig- 
nals from the input Image by detecting coarse scale edges; (c) nrxxdifying the detail signals according to the contrast 
gain signals; and. (d) adding the modified detail signals to the coarse signals to obtain an output Image. 
3s [0009] An object of this invention is to provide an effective means for compressing the dynamic range of an input 
image so that It can be properly rendered to an output display macfium that has a narrower dynamic range, without the 
deficiencies off the prior arts as mentioned above. 

[001 0] Another object of this invention is to provide a means for automatically computing an Image-dependent torie 
scale (gradation) curve for dynamic range compression. Still, another object of the invention is to provide a means for 
40 computing gain-control signals for suppressing the edge banding artifacts around high-contrast edges. Yet another 
object is to provide a general and flexible image decomposition that allows explicit extraction of Important image details, 
such as edges, lines, and textures. 

BRIEF DESCRIPTION OF THE DRAWINGS 

45 

[0011] 

Figure 1 shows how the banding artifact is generated by prior arts; 

Figure 2 shows one emtxxjiment of this invention. The output image is reconstructed from the weighted average off 
so the decomposition filter outputs from the original image and its tone scaled version; 

Figure 3 is a block diagram of the prefen-ed embodiment of this invention. It shows the three msyor modules that 

function together to produce a dynamic range compressed output image ff,ut\ 

Figure 4 is an expansion of Rgure 3 with nmre details of the filter banks module; 

Figure 5 is a block diagram of the tone scale curve (TSC) module; 
55 Figure 6 shows the characteristic curve of an example photographic color paper. The diagram also shows the black 

point the white point, the black value, and the white value; 

Figure 7 shows the parameters used in the tone scale curve construction; 

Figure 8 shows how an interval of the slope curve is interpolated from the nodes; 
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Figure 9 shows an example of the histogram (the bottom curve) of an input image and the tone scale curve con- 
structed from the histogram; 

Figure 1 0 shows the ideal wa/ to compress an illumination edge, leaving all the reflectance edges alone; 
Rgure 1 1 shows the details of the contrast gain-control module; 
6 Rgure 1 2 shows the various low-pass (smoothing) L filters that are used on images of different sizes for computing 
the contrast gain-control signals; 

Rgure 1 3 shows the preferred embodiment of the filter banks (FB) module: and, 

Rgure 14 shows a comparison of the input signal profile of a line through the input image / and the output signal 
profile of the same line through the output image /^uf. 

10 

DETAILED DESCRIPTION OF THE INVENTION 

[0012] An overview of the invention is described with reference to Fig. 2 and Fig. 3. An input image. /, with wide 
dynamic range can be considered as having a component A which contains the global, large-scale dynamic range sig- 

15 nal, and another component B which contains the local, fine scale image details. This can be written as / A + H . TTie 
task is to compress the dynamic range of A by applying a compres^ve tone scale (T) to it. For purpose of enhancing 
the Image details in 5, an image enhancement operation E to B can be applied. Therefore the processed output image, 
can be written as /' = T{A) + E{B) . Unsharp masking in the prior arts uses low-pass filtering or weighted local aver- 
aging to derive the A component from the input image, and compute S as S = / - A . In this invention, the separation 

20 of the A and B components from / is through a decomposition filter bank. TTiese fitters need not be linecu*. For example, 
since edges are Important Information in images, the decomposition can be accomplished through edge detectors at 
many spatial scales. On the top half of Fig. 2, the Input Image Is past through the filter bank 1 0. On the lower half, the 
input image is input through a tone scale processor 40 where it is compressed before it is past through the decomposi- 
tion filter bank 20. If the dynamic range of the input Image is small, the tone scale curve will not reduce the dynamic 

25 range further and the output from the upper filter bank will be essentially the same as that from the lower filter bank If 
the dynamic range of the image is large, then the output from these two filter banks will differ significantly, especially in 
the highlight and shadow regions where the tone scale curve will try to compress. The central question is to decide how 
to best combine the outputs from the two filter banks so that the reconstructed output image will have a narrower 
dynamic range to match the Intended output medium, but also have all the useful Image details properly rendered so 

30 that they are pleasingly visible. Tl^s invention discloses a contrast gain-control means 30 to produce the desired weight- 
ing factors, W and to combine the outputs from the two filter banks 10 and 20. The output from the decomposition 
filter bank 1 0 is always weighted more in the low-modulation areas in an image, i.e., W» W in the areas where high 
contrast edges occur, the output from the decomposition filter bank 20 is weighted more ( VV » W), and the output image 
Is reconstructed by a reconstruction filter bank 50. The reconstructed output image lout wiQ take its high contrast edge 

ss signals from the dynamic range compressed image. The resulting images will have the fine low-contrast details as in 
the original Input Image ard also have the much-reduced high contrast edges as in the dynamic range compressed 
image. 

[001 3] The Invention as expressed In Rg. 2 can be performed more efficiently by a different embodiment as shown in 
Fig. 3. Instead of processing the original version of the irput image and Hs tone-scaled version through the decompo- 

40 sition filter bank separately, the new embodiment only processes the difference between the two versions. Therefore, 
the function of the decomposition filter bank 60 and the reconstruction filter bank 70 can be treated as extracting the 
desired image details or other features that might have been lost in the tone scale compression of the input image. This 
new perspective Is quite Inportant t)ecause the tone-scaled image Is considered as the baseline output Image that has 
the desired dynamic range and then the algorithm can choose freely what image details is desired to add badk to that 

45 baseline output Image. This means that the design of the decomposition and the reconstruction filter banks 60 and 70 
has to satisfy fewer constraints than a perfectly analysis-synthesis filter pair has ta The present invention will focus on 
the framework laid out in Fig. 3, with the understanding that Fig. 2 is an alternative embodiment using most of the same 
Ideas about the tone scale curve design and the contrast gain-control with between-scale interactions. 
[0014] Still refeiTing to Fig. 3. this invention consists of three major modules: (1) tone scale curve (TSC} module 80, 

so (2) contrast gain-control (CGC) module 90, and (3) filter banks (FB} module 100. The function of the TSC module 80 is 
to generate a tone scale curve that will map the dynamic range off the input image into the dynamic range of the output 
display media. The CGC module 90 is to generate the banding artifact suppression signal to nxxlulate the edge signals 
extracted in the filter banks (FB) nrxxiule 1 00. 

[0015] An input image / is processed through the (TSC) module 80, which computes a tone scale curve for the input 
55 image and maps it through the curve to produce a tone-scaled (dynamic range compressed) image /*. The tone-scaled 
image /' is then subtracted from the input image / to produce a difference image D. The difference image D is proc- 
essed through the filter bank 60. which extracts the edge and detail signals from the difference image D. The edge sig- 
nals are modulated by the contrast gain-control signals from the (CGC) module 90 that uses edge signals from all 
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scales to generate conrtrast gain-control signals. lYie output D'from llie filter bank 70 therefore contains edges and 
details in the difference image D, with the edge banding artifacts suppressed by the contrast gain-control signals. D' is 
then added to the tone-scaled image /' to produce the output image, lout- 

[001 6] The block diagram of the invention in Figure 3 is expanded further in Figure 4, which shows the detailed ctruc- 
5 ture of the filter banks nrxxlule 100 (see Fig. 3) and how the gain-control signals are used to modulate the filter bank 
signal processing. The overall image processing steps are described and then they are followed by the detailed descrip- 
tion of each of the three main modules. 

[0017] Rrst. an input image is processed in the tone scale curve (JSC) nKxIule 80. The dynamic range of an input 
image. /. is determined from its histogram. A tone scale cun^e Is then constructed to oompress the dynamic range of 

10 the input image to that of the intended output display media. The input image is then mapped through the tone scale 
curve to produce a toned-scaled image. The difference image D between / and I' (i.e., D = is then passed 

through a filler bank 60. consisting a set of N decomposition filters. F^, and F/v 60a, 60b, ... 60n. These fillers 

are feature extractors and they can be either linear or nonlinear Their main function is to extract the image features, 
such as edges, tGDdures, and shadings, that are considered as desirable targets for enhancement or suppression in a 

75 given application. The output through D^, from each of the filters is modulated in Modulator through h/l^ (1 10a 
through 1 1 0n) by a gain-control signal which is produced by the contrast gain-control generator 90 finom the input image. 
I and all the outputs from the decomposition filter bank 60. Each of these gain-adjusted signals. D'i through D /v is then 
passed through a reconstruction filter bank 70 consisting of a set of filters H^^ through {70a through 7Dn). The 
outputs from the reconstruction filter bank 70 are then summed together to produce a new difference signal, D\ which 

20 is IhOT added back to the tone-scaled image. to produce the desired output image, /^^ = /' + D\ 

[001B] Images can be captured and digitized by various imaging devicas. It is important to calibrBte the digitized 
Image to the desired mertric. In the following description of this Invention, all images are calllmted to relative log scene 
exposures (BLSE). meaning that the log scene exposure has been shifted so that when it Is printed digitally, the bright- 
ness of the image looks pleasing. For convenience, the relative log scene exposure is scaled up by a factor of 1000 so 

25 that one can use only integer values to represent the input and the output images. However, the intermediate results 
from the various filters are represented as floating point numbers, so that proper precision is always retained. 
[0019] Having described how the overall system works, each module will now be described in detail. 

The Tone Scale Curve (TSO Module: 

30 

[0020] Figure 5 shows a diagram of the Tone Scale Curve (TSC) Module 80. This module 80 computes a unique tone 
scale curve for each input image and applies rt to produce a tone-scaled version of the input image. The msuor function 
of the tone scale curve is to compress the large dynamic range of the input image into a smaller dynamic range so that 
the output medium can render the image effectively. 

55 [0021 ] The Tone Scale Module 80 usee a histogram accumulator 1 20 to compute the image histogram from the input 
image. From the histogram, the module 130 oomputes the aim, the dark value and the bright value of the Input Imaga 
The highlight slope. Sh. the mid-tone slope S^. and the shadow elope are then computed in block 1 40. Firom these 
computed parameters, a tone scale curve is generated In block 150. The tone scale curve is implemented as a tookup 
table in 1 60. The input image is mapped through the lookup table 1 60 to generate the tone-scaled image 

40 [0022] In order to adjust the dynamic range of the input image so that it can be effectively rendered to the intended 
output medium, the dynamic range of the output medium is needed. This information Is available once an output 
medium is selected. For example, if the output medium is photographic paper, one can look up the minimum density 
(Dmin) and the maximum density J of the paper and Its dynamic range Is appraxlmately D^ax - ^mtn- To account 
for viewving flare and to allow some rooms for signal modulation in the bright highlight and deep shadow, a slightly 

45 reduced dynamic range than D^ax "^min consWered. For example. Figure 6 shows a characteristic curve for a pho- 
tographic paper. The x^is is the log soene exposure and the y-axis is the resufting status A density after the paper is 
processed and developed. In digital printing, the printer can be calibrated to produce a photographic print according to 
such a characteristic curve. The minimum density of the paper is 0.05 and the maximum density is 2.34. As mentioned 
before, in order to anew for signal nfKXfulation, a density of 0.1 26 is selected as the white point on the paper and 2. 1 1 2 

50 as the black point, resulting In a slightly reduced dynamic range of 1.986 = 2.112 - 0.126. Because an image can be 
printed either lighter or darker by shifting the log exposure on the x-axis to the r^ht or to the left, a density balance point 
needs to be specified where the middle gray in the image shall be printed in order to get a pleasing brightness rendition. 
This density balance point is referred to as the aim point of the image. The aim point is to be printed to a paper density 
0.8. The Ijlack point and the white point on the y-axis of the paper curve in Figure 6 are mapped to the black value and 

55 the white value on the x-axis, that represents the relative log scene exposure. The resulting black value and white value 
are {aim - 800) and {aim + 640). For an input inrtage, a density balance value, i.e. the aim. has to be determined by a 
density tialance algorithm, which is typically provided in almost all photof inishing printers. Once this aim value is cho- 
sen, the available dynamic range for the ou4)ut Image is restricted to the range from [aim - 800) to [aim + 640). There- 
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fore, as shcfwn in Fig. 7, the available paper dynamic range has to be split into the white far»ge [aim to aim + 640) 
and the black range i^im to aim - 800). TTiese two ranges are defined relative to a given display medium and Itiey 
are fixed once an output medium is selected. If the input image has pixel values that etxceed this range. It Is the Job of 
the tone scale curve to compress Ihem into this range by adjusting the slopes in the various portions of the curve. The 

5 following describes how ttie tone scale curve is constructed to accomplish this dynamic range compression task. 

[0023] Rrst, the TSC module 80 computes the histogram of an input image. The upper 99.5 percentile code value of 
the Input histogram is tentatively designated as the bright value of the image, and the 0.5 percentile code value, the dark 
value of the image (see Rgure 7). The upper range H„ of an image is defined to be from the aim of the image to itie 
bright value of Ihe imaga The lower range Rf of an Image is from the aim of the image to the dark value of the image. 

10 ThesB two ranges, Ru and R\, vary from image to image, while the other two ranges. Ry„ and R^, are fixed once an out- 
put medium is chosen. Therefore, the job of the tone scale curve is to map R^ into f?y,^, and Rj into R^- There are infi- 
nitely many tone scale curves that can be constructed to do this mapping. However, the following method of 
construction is the preferred embodiment for this invention. 

[0024] Refening to Rgures 6 and 7, the tone scale Is contructed to satisfy six constraints: (1) the dark value is mapped 
75 to the black value; (2) the slope at the dark value is specified by the shadow slope S^; (3) the aim value of the input 
Image is mapped to itself; (4) the slope at the aim value is specified by the mid-tone slope S^; (5) the bright value is 
mapped to the white value; and. (6) the slope at the bright value is specified by the highlight slope, S^. 
[0025] Mathematically, this is a problem of inteipolation under the constraint that the curve has to pass through 3 
given points at given slopes. Before describing how such a curve can be constructed, how the points and slopes are 
20 determined from a given image is first specified, f^rst. the lower range Rf of the input image is compared with the black 
range, R^,, of the output medium. If Rf Is larger than Ru, (meaning tone compression is needed In the shadow), then the 
0.5 percentile value of the histogram is taken as the dark value. Otherwise, the input value [aim - 800) Is taken as the 
dark value. The shadow slope at the dark value in either case is always set at a value larger than 1.0. The preferred 
shadow elope is 2.0. Next, the upper range R^ of the input image is compared with the white range, R^^ of the output 
25 medium. If fly is larger than Ry^^ (meaning tone compression is needed in the highlight), then the 99.5 percentile value 
of the histogram Is taken as the bright value, and the highlight slope S^^ is enrpirlcally set acooidlng to the following for- 
mula: 

30 ^^-^'^*h^'w!i^' 

If Ru is equal to or smaller than R^ then the input value {aim + 640) is taken as the bright value and the highlight slope 
is set to 1 .0. If an algorithm is available for estimating the scene contrast of the input image, the mid-tone slope 5^ 

35 can be adjusted accordingly If such an algorithm is not available, then the preferred value is » 1 0. 

[0026] There are many curves that can pass through three given points (the dark value, the a/m. and the bright value) 
at the specified slopes (S^ . S^, and S/J. Additional constraints are needed to construct a unique curve. For tone scale 
purpose, the instantaneous slope at any point on the curve is required to be greater than a minimum value S^^i 
smaller than a maximum value, S^wc as specified by the user. This is to ensure that excessive quantization due to too 

40 high or too low slopes will not occur It is further required that the slope changes at the three given points have to be 
gentle and the effect of a specBied slope decays whh distance as a Gaussian function with a specified standard devia- 
tton. 

[00271 l^a problem is formulated In a more general setting. Given S^i„, Smac* points. (Xj.y/). / = 1 . 2. ....n, where X/ 
< xj and /, < yy. for all / < y, and n slopes. ^ S/ s 5^, / = 1. 2, .... n . a monotonic curve y = f{x) , is needed such 

4S that y/ = /(>/). Sf = r[Xf) for / = 1. 2 n. and S^n ^ r(x) ^ S^ax ^- Th« problem Is solvable only If 5^n(^y - 

^ii^iyj- Yi)^ Smax (^/ - >0) fo'' a\\ j > /. This cun^e construction problem is solved in this invention by a procedure to 
compute f(x) first and then the curve y = f[x) is constructed by integrating fXx). The integration is started from the aim 
point and completed in two directions, one going up stopping at the bright value, and the other going down stopping at 
the dark value. In the process of integration, if the focal slope at any point is greater than S^ex* >t is set at S„mx- it is 

so smaller than S^jp. it is set at S^i^. 

[0028] To solve the afciove curve constaictlon problem, two more constraints are imposed so that the solution can be 
efficiently derived: (1) each interval is interpolated independently, and (2) the interpolation function is a Gaussian func- 
tion with a user-specified standard deviation (called width of influence). Figure 8 shows how r(x). the first derivative of 
the tone scale curve, is constructed by Gaussian interpolation functions. Since each interval is interpolated independ- 

55 ently, the interpolation process from to Xz will be discussed without loss of generality. If it is assumed that the width 
of influence at x^ and X2 fe a, and 02, respectively The interpolation function has the following form: 
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2 2 

9lx) = A exp[- ^i:^] + a e>cpl- + 0. 



(2) 



and the three unknown coefficients A, B, and O can be solved from the ix»undary conditions: 



10 



2a9 



2a: 



75 



20 



J = ^U2)■^^^l) = /2■/1• 



The follGwing quantities are defined as: 



a = exp[ — 1 

2a I 
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2a? 



3S 
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J «p[- ^^^]dx 



2a? 

•i 

'1 



2 

2ao 



Then the solution for A, S, and C can be expree&ed as: 



45 



-A- 




8 




.c. 





1 Ti r ^'c*i) ■ 



r1 a 1 T 



(3) 



so It is possible that Hie matrix can be singular and there exists no solution for the particular constraints. The algorithm 
then reduces at and 02 ^f*^ ^ solve the equations again. This cycle will repeat several times until a^ and <^ 
become too small and the algorithm simply defaults to a constant slope solution for the interval. The preferred values 
of CT at the three given points are cr^ = 1 00.0 for the shadow, 05 - 300.0 for the mid-tone, and 03 = 80.0 tor the highlight 
Figure 9 shows an example of the tone scale curve constructed by the method in this invention. Tlie noisy curve at the 

£5 bottom is the histogram o< the input image. 
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The Contrast Gain-Control (CGC^ Module 

[0029] A large dynamic lange of a natural acene is almost atwaye caused by illumination differences in various parts 
of the scene. The boundary between two different illunrunaticns is called the illumination edge, which is often of high 

6 contrast and high exposure gradierTt In connparison. the boundary between two different reflecting materials is called 
the reflectance edge, which is often of lower contrast and smaller exposure gradient An idea! way to compress the 
dynamic range of an Image is to reduce the magnitudes of the ifiuminatlon edges, while presenfing all reflectance 
edges. Rgure 10 illustrates how this ideal compression operatbn should be performed. An important step to achieving 
such selective edge magnitude compression is to identify where the illumination edges are in the image. Once the niu- 

10 mination edges are located, the magnitude of the edges can be controlled by adjusting the contrast gain-control applied 
to the iiluminalion edges. The function of the Contrast Gain-Control (CGC) Module is to detect all the illumination edges 
in the input image and generate a gain-control signal for each pixel of the image. These gain^oontrol signals are applied 
to the fOter banks whose job is to extract edge signals of all magnltudis. TTie small-magnitude reflectance edges are 
given a contrast gain of 1.0 or higher, while the large-amplitude Iiluminalion edges are given a very small gain. As a 

IS result of this gain control mechanism, the illumination edges are greatly suppressed in the output, D\ of the filter banl<s 
(see Rgure 4). Ideally. D'^OaX pixels where illumination edges occur. Therefore, the illumination edges in the final out- 
put Image, /oy,. are mainly from the tone-scaled version of the input image, which is. o1 course, properly compressed in 
its dynamic range. Mathen^atically, i^^f^r + D'^rior illumination edges. In comparison, the reflectance edges in the 
difference Image D pass through the filter banks virtually unattenuated. Therefore, the output D'from the fitter banks 

20 module contains almost the exact copy of the reflectance edges in I - r, and when it is added to /' to produce foot* 
refleclance edges are well preserved In the output imaga Mathematically, /^u/ -i- D' + = / for reflectance 
edges. 

[0030] One important fact in image formation is that illumination edges are not easily distinguishable from reflectance 
edges gA a small spatial scale because certain reflectance edges also have large edge magnitudes when edge magnl- 

25 tudes are computed from small sfxaXlaH distances. A reliable detection of illumination edges has to be performed over 
large spatial distances. This is a very inportant point and it means that the contrast gain-control signal has to be oonrv 
puted from a coarse scale edge detection. Any scheme that attempts to suppress the banding artifacts has to use a 
coarse scale edge detector to control the contrast gain factors for edge filtering. This is why a gain-control function gen- 
erated internally within each resolution can not work well. The gain-oontrol signal for the fine resolution details has to 

30 come from the coarse resolution edges. 

[0031] Figure 1 1 shows the preferred embodiment of the contrast gain-oontrol signal generatioa The input image is 
filtered through a low-pass (smoothing) filter to remove most of the fine reflectance edges. The low-pass fitter {L) 170 
in Figure 1 1 is related to the fitters of the coarsest resolution F^ 60n in the filter t>anks (FB) GO in Figure 4. In tfie pre- 
fen-ed embodiment A/ » 6, botti F5 and Fs have the same coarse resolution. ITie common lowi3ass filter in F5 and Fe 

3s is S(av}S(<oy)S(2a>jr)S(2cDy) and the filter L should be ch^n to have a comparable coarse resolution. A simple choice - 
Is to set Hm^.my) = S{<£>^)S{Oy)S[2<s^x)S{2iOy) . TTie filters F5, Fe. and S In the preferred embodiment wHl be 
described in detail shortly in the context of tine fitter banta module. Figure 12 shows a series of smoothing filters that 
can be used for various image sizes. The filter Is generated from a basic kernel of [1 ,4,6.4. 1] with multiple convolutions 
with itself. In order to preserve the zero frequency response, the area under each smoothing filter is normalized to 1 .0. 

40 For practical implementation of the invention, the exact shape of the smoothing filter is not important as long as it is 
smooth and compact For example, a truncated Gaussian filter is also quite acceptable. 

[0032] The algorithm then computes the edge gradient on the low-pass filtered Image by the gradient amplitude proc- 
essor 180. TTie edge gradient amplitude is computed by taking the square root of the sum of the square of horizontal 
pixel difference and the square of vertical pixel difference. The pixel difference is computed over a distance proportional 

45 to the size of the smoothing fitter. The computed edge gradient amplitude is Ifien mapped Itirough a function go 

the output, V, is used to modified Itie gain-control signals G^, G2 Gf^. The general functional form of is chosen 

so that when the edge gradient amplitude Is small, its output value V is 1 .0 and when the edge gradient amplitude is 

large, V, is zera TTie \/ value is used as a mask image so that the functions g-i, gz gn (200a. 200b ... 200n} can 

adjust its gain spatiaDy When the edge gradient amplitude is large, V is zero and the gain-control functions. Qi, will 

BO reduce the local gain factor to sippress the edge banding artifacts. The N additional input signals to the Contrast 

Gain-Control module are the output Di.Da. D,^, from the decomposition filters F-1.F2 Fy^. These input signals are 

first mapped through d^,d2, df^ {210a, 210b ...210n) to calculate their energy For example, a simple squaring func- 
tion, d f(x) = , can be used. These N energy signals are then used to produce their weighted linear combinations via 

an NxN matrix 220. The N output signals, ,02. ■ , C/v, from the matrix are further modified by functions g-^ ,g2 

On* (200a. 200b ... 200n) under ttie influence of V, to produce the final gain-oontrol signals G-i ,62 Gfj. The general 

form of the functions. g^^2^ .... gf^, (200a. 200b ... 200n) is similar to that of go(190). TTiat is, g\ produce gain factors 
dose to 1 .0 when Is small. However, when Cf is large, the output of g, depends on l^. If is close to 1 .0, than G^ is 
also dose to 1 .0. If is close to zero^ then G/ Is suppressed to be near zero as wefl. 
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[0033] The operation of the GGC module can be better understood from ihe following exampla The illumination edges 
ueuaDy have large gradient amplitudes at a coarse spatial resolution as determined Itie lowiaassfifter L in Rgure 1 1. 

The value V is ten very small. As a consequence, all the galn-conlrol signals. G^,G2 G/v> are also set at a very small 

value if Ci.Ca On are large, and Ihe illumination edge is suppressed. In comparison, reflectance edges are mostly 

5 smoothed out by the smoothing filter i (1 70) and only have small gradient amplitudes. Therefore. V will be close to 1 .0. 
As a consequence, all the gain-control signals, Gi .Ga. Gj^, are also set at a value close to 1 .0. and the reflectance 
edges In Di,D2. .... of Fig. 4 are passed without any reduction In magnitude. It is worth pointing out again that this 
capability to discriminate between the illumination edges from the reflectance edges comes from the low-pass filter, L 
(170). as well as the amplitude mask produced by Qq (190). 

10 

The Filter Banks fFB^ Module 

[0034] Hie Filter Banks (FB) Module (see Figure 4) receives the difference image D = i - i\ which represents the 
image details in the original image / that are lost in the dynamic range compressed image These details include both 

15 the fine reflectance edges and the coarse illumination edges. The task of the filter banks is to extract the high frequency 
details, and to remove the very low frequency (DC) component that constitutes the mgyor portion of the large dynamic 
range in the input imaga But, not all the high frequency details should be allowed to pass through the filter banks. The 
part of high frequency signal that is associated with the illumination edges should be suppressed and the suppression 
is achieved by the gain-control signals from the CGC module. 

20 [0035] The number of filter paths, N, is determined by the cost and system througlput constraints. As the number of 
filter paths increase, the banding artifact raduces, because more frequency channels allow the illumination edges to be 
suppressed more successfully as a function of spatial scales. Referring to Figure 4, one of the unique feature of the filter 
banks in this invention is to separate each fitter path into two parts, F,- and H/, with a nonlinear gain-control in between. 
This design is desirable because the filter 5 is designed to be edge (or texture} detectors in different spatial scales and 

25 orientations. Therefore the from filter Fj represents edge signals at a particular scale in a particular spatial orientation. 
It is this explicit edge contrast signal that is modulated by the contrast gain-control signal. Since the optimal decompo- 
sition by Ff into edge representations does not necessarily lead to an optimal reconstruction by simple summation, a 
post processing filter. H/, is required to reconstruct the difference image back. (Mathemalicany, F/ filters are not orthog- 
onal.) There are many possible choices for the decomposition-reoonstruction filter pains. The nunrt)er of filter paths, N, 

30 and the image size win constrain the choices to a few for an efficient implementation. In the preferred emkxxfiment, three 
frequency bands and two spatial orientations (vertical and horizontal) are selected, making the total number of paths 
equal lo A/ « 3 x 2 « 6. Let a»r and ooy be the spatial angular frequency in the x and y directions. The filters have the 
folkmnng frequency 





(4) 




(5) 




(6) 


H2(o>,.a»j,)= L(a>,)0(a>p 


C7) 


F3(a>,,i»y) =Sfa»x)S{<Dy)fl(2a>,) 


(8) 


H3(®x.®y^ = P(«;,)P(<«>y)Q(2« Ji.C2a>j,) 


(9) 


F^{<a^,ay} = S(a,)Sl<o^R&ay) 


(10) 


H^{fo^.fOy) = Pra»,)P(a>y)L{2<a,)Qp«(»p 


(11) 


FgCo^.oap = S(«jf)S{(BpS{2a>,)S(2t0j,)ff(4oijf) 


(12) 




(13) 


F^i<b^.my) = S(a>^)S[<Dy)S{2<D^)S[2ayy)R[*a^) 


(14) 


Hi{fi^.<s^y) = P(<,i^)P!ia>y)Pl2a^)P(2iOy)HAa,^)0{Aci>y) 


(15) 
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where 



S(®)=C08*(|) 



(16) 
(17) 



10 



IS 



20 



P(fl>) = S{a>) 
Q{iD) = [1-S^(ffl)]/fl{a>) 



L(<s) [1 + S^(<»}]/2 



(18) 

(19) 
(20) 



These fillers are from the family of edge-vwavelet filters that were used in Mallat and S. Zhong's paper, "Characterization 
of signals from muWscala edges," iEEE Transactions on Pattern Analysis and Machine Intelligence, 14, 7, 710-732. 
1992. The fitter banks can also be implemented in a different form as shown in Rgure 13, where Hie simplified notations 
have been used to make the figure more readable. For example, SxSy{ai) denotes S(cDjJS((Dy), Rx{2<o) denotes 
H(2a)jf). and so on. It should be pointed out that these filters are not the only choices. The preferred choice is that the 
Ff filters should extract edge-like information. 

[0036] From the definilion of Fourier transform of a sarrpled functbn f{n): 



25 



(21) 



30 



/(n)o^ J F(fl>)e**"db. 



(22) 



the filter coefficients are derived as: 
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Producing the Outout Image 

[0037] The output signal D'from the filter banks contains all the fine details that are in the original input image / , but 
are missing In the ton»-scaled and dynamio-range-compressed image t\ One of the most important features of this 
invention is that the high contrast illumination edges are suppressed by the contrast gain-control and Hierefore they are 
not present in D\ the output of the filter banks nx>dule. By adding D'back to /' , the algorithm produces an output image 
fout that is free from the barxiing artifacts in the dynamic range compressed Image. Figure 14 shows a comparison of 
the input signal profile of a line through the input image / and the output signal profile of the same line through the output 
image /^uf • The dynamic range of the input image signal is greatly reduced and yet. there is no (sanding artifact at the 
very high contrast edges (anound pixel locations 160 and 170). The low-contrast image details at other locations are 
well preserved. 
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Claims 

1. A method of reducing the dynamic range of an Input image for effectively rendering the input image on an output 
display medium, the method comprising the steps of: 

5 

(a) extracting detail signals and coarse signals from the input image; 

03) generating contrast gain-control signals from the input image by detecting coarse scale edges; 

(c) modifying the detail signals according to the contrast gain signals; and, 

(d) adding the modrfied detail signals to the coarse signals to obtain an output Image. 

10 

2. The method as in daim 1 , wherein step (b) comprises the steps of: 

(b1) processing the input image through a low-pass filter to obtain a blurred version of the input image; 
(b2) computing gradient amplitudes of the blurred image; and. 
75 (b3) using the gradient amplitude as an Index Into a look-up table to generate the contrast gain control signal. 

3. The method as In claim 2, wherein the step of extracting the detail signals Includes processing the input image 
through a plurality of band pass and high pass filters. 

20 4. The method as in daim 1 further comprising modifying the coarse signals with a tone scale curve after step (c). 

5. A method of reducing the dynamic range of an input image / so that it can be rendered effectively on an output dis- 
play medium, comprising the stepe of: 

25 (a) constructing a tone scale curve T from the input image /; 

Oa) mapping the input Image / through a tone scale curve Tto produce a tone-scaled image /' = T(f) ; 

(c) produdng a difference image D = i-Vtti subtracting the tone-ecaled image from the input image; 

(d) producing a modified difference image, D\ by processing the difference image D through a plurality of 
band-pass and high-pass fOters; and, 

aa (e) adding the modrfied difference image D' to the tone-scaled image /' to produce a dynamic range com- 

pressed output imaga 

6. The method according to claim 5, wrherein Ihe step (a) of constructing a tone scale cun^e comprises the steps of: 

35 (a1) computing the histogram from the input image; 

(a2) finding the low end. the median point, and the high end of the input values from the said histogram at some 
fixed percentiles; 

(aS) mapping the low end point to the black level of the output medium with a higher than normal local slope; 
(a4) mapping the median point to the medium gray level of the output medium with a normal contrast; and, 
40 (a5) mapping the high end point to the white level of the output medium with an adjustable contrast, depending 

on the shape and the range of the upper tali of the said histogram. 

7. Tlie method according to claim 5. wherein the band-pass and high-pass filters used In step (d) are edge detectors 
at various spatial ecales. 

8. A method of generating a tone-scale curve from an image for dynamic range compression, comprising the steps of: 

(a) computing a histogram from the Irput image; 

(b) finding a low end, a median point, and a high end of the input values from the histogram act some fixed per- 
so centiles; 

(c) mapping the low end point to a black level of the output medium with a higher than normal lo^ slope; 

(d) mapping the median point to a medium gray level of the output medium with a normal contrast; and, 

(e) mapping the high end point to a white level of the output medium with an adjustable contrast, depending on 
the shape and the range of an upper tail of the histogram. 

55 

9. A method of image contrast enhancement using contrast gain-control signals for suppressing banding artifacts, 
comprising the steps of: 
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(a) decomposing an input image into edge gradients of several spatial scales; 

(b) generating contrast gain-control signals G from the original input Image; 

(c) enhancing each edge gradient signal by modifying its amplitude according to separately generated contrast 
gain-control signals G in step (b); and, 

(d) reconstructing an enhanced oulput image from the modified edge signals of step (c). 

1 0. The method according to claim 9. wherein the step (b) of generating contrast gain-oonrtrol signals G comprises the 
steps of: 

(b1) low-pass filtering the input image; 

(b2) computing an image gradient fnom the low-pass filtered image; and» 

(b3) mapping the magnitude of the image gradient through a lookup table which Is calculated from a smooth 
monotonic function to produce the contrast gain-control signals G. 
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